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LONG-TERM  GOALS 

The  primary  goal  of  this  project  is  to  develop,  on  the  basis  of  the  present  SWAN  benchmark  suite,  a 
first  version  of  a  generic  community  suite  covering  both  oceanic  (unsteady  state),  time-dependent)  and 
coastal  (steady  state)  conditions.  This  will  provide  the  basis  for  a  suite  that  will  qualitatively  and 
quantitatively  compare  the  computational  results  of  any  wave  model  of  the  spectral  type  with  objective 
data  from  theoretical  sources  (analytical  solutions)  and  empirical  sources  (laboratory  and  field 
observations).  The  suite  will  be  generic  in  the  sense  that  a)  a  diversity  of  wave  models  can  be 
evaluated  against  such  data  and  that  b)  an  internationally  agreed-upon  fixed  set  of  tests  is  used  so  that 
the  score  of  different  models  can  be  intercompared.  The  suite  is  a  community  tool  and  will  be  provided 
to  the  international  community  to  enable  operational  institutes  and  researchers  around  the  world  to  test 
their  wave  models  in  a  joint  framework  with  common  data  and  common  score  definitions. 

The  SWAN  benchmark  suite  which  is  mentioned  above  has  been  developed  over  the  last  few  years  as 
a  steady-state  benchmark  suite  (Ris  et  ah,  1999;  WL  |  Delft  Hydraulics,  2000)  to  assist  operational 
institutions  and  researchers  to  evaluate  the  scientific  and  numerical  performance  of  (new  versions  of) 
the  SWAN  coastal  wave  model  (Booij  et  ah,  1999).  In  this  suite  the  model  performance  is  evaluated  by 
comparing  the  computational  results  with  data  in  academic  cases  (analytical  solutions  and  laboratory 
data)  and  in  real  field  cases  (sandy  coasts  including  the  surf  zone). 

SCIENTIFIC  OBJECTIVES 

The  main  objective  is  a  generic  system  that  can  evaluate  numerical  wave  models  to  support  the 
development  of  the  best  possible  wave  forecasting  (and  hindcasting).  This  requires  the  evaluation  of  a 
variety  of  wave  models  in  a  variety  of  conditions  ranging  from  validation  tests  (academic)  to 
verification  tests  (real  field  cases)  and  from  large-scale  (oceanic)  conditions  to  small-scale  (coastal) 
conditions.  With  the  proposed  development  of  a  community  suite  with  an  internationally  agreed-upon 
set  of  generic  tests  cases  and  high  quality  data,  this  evaluation  can  be  carried  out  efficiently  for  any 
wave  model  of  the  spectral  type.  The  tests  will  be  standarised  such  that  the  score  of  one  model  can  be 
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compared  to  that  of  another  model.  This  development  avoids  the  typically  required  ad  hoc  efforts 
involving  extensive  testing  of  two  or  more  (versions  of)  models  against  observational  data. 

APPROACH 

To  facilitate  the  evaluation  of  a  large  diversity  of  wave  models  of  the  spectral  type  in  oceanic  and 
coastal  applications,  Delft  Hydraulics  will  make  the  currently  available  suite  of  steady-state 
benchmark  tests  of  the  SWAN  model  generic  and  it  will  extend  it  to  unsteady-state  (oceanic) 
conditions.  It  will  be  made  generic  by  providing  fixed  formats  and  example-converted  input  files.  It 
will  be  extended  with  one  available  steady-state  US  field  data  set  (i.e.  observations  of  DUCK- 
DELILAH)  and  by  adding  up  to  eight  unsteady-state  data  sets  (primarily  from  US  sources).  The 
selection  of  the  tests  will  be  made  in  consultation  with  the  WISE  group  and  subject  to  the  approval  of 
ONR  representatives. 

Within  the  project  the  steady-state  (coastal)  model  STWAVE  (see  e.g.  Smith  et  al.,  1999)  will  be 
added  to  the  steady-state  benchmark  suite  and  the  unsteady-state  (oceanic)  model  WAVEWATCH-III 
(Tolman,  1991,  1998)  will  be  added  to  the  unsteady-state  benchmark  suite.  The  two  versions  of  the 
suite  (steady-  and  unsteady-state)  will  be  fixed  in  the  sense  that  no  options  will  be  available  to  the  user 
so  as  to  prevent  deviations  from  the  definitions  of  the  benchmark  scores.  On  the  other  hand,  the  code 
will  be  open  enough  that  new  or  modified  test  cases  and  new  or  modified  statistical  scores  are  readily 
added  (minimum  flexibility  for  the  operational  use  and  maximum  flexibility  for  continued 
development).  Executing  the  benchmark  suite  for  a  specific  wave  model  will  only  require  a  conversion 
of  all  I/O  files  for  that  model.  Two  such  I/O  conversions  will  be  provided:  one  steady-state 
(STWAVE)  plus  one  unsteady-state  (WAVEWATCH-III). 

The  comparison  of  the  computational  results  with  observations  in  this  suite  will  be  qualitative  through 
the  use  of  graphics  and  it  will  be  quantitative  by  using  statistical  measures.  The  wave  parameters  to  be 
considered  are  the  significant  wave  height  Hs,  the  peak  wave  period  Tp,  the  mean  wave  periods  Tmoi 
and  Tm02,  and  frequency  spreading  <ja  for  the  steady-state  test  bed  suplemented  with  the  (overall)  mean 
wave  direction  9  (if  available  in  the  observations)  and  the  directional  spreading  9a  for  the  unsteady- 
state  test  bed.  The  statistical  measures  are  the  bias,  the  rms-  and  mae-error,  the  scatter  index  (except 
for  directions),  the  operational  performance  index  and  the  model  perfonnance  index  (if  relevant).  Brief 
documentation  will  be  provided  in  the  form  of  limited  guidelines  for  experienced  users.  This  will 
consist  of:  a)  implementation  and  user  manual,  b)  a  description  of  the  suite  of  bench  mark  tests  and  c) 
reporting  protocol. 

The  benchmark  tests  (steady-state  and  unsteady-state)  will  be  executed  and  the  results  will  be  reported. 

The  work  will  be  carried  out  in  close  consultation  with  several  leading  model  developers  and  users. 
Active  support,  assistance  and  advice  on  the  project  is  provided  by  L.  Holthuijsen,  N.  Booij  and  H. 
Tolman. 

WORK  COMPLETED 

•  The  steady-state  test  bed  has  been  completed  and  has  been  made  available  to  interested  researchers. 
Until  the  end  of  September,  the  test  bed  (3  CD-ROMS,  a  manual  and  two  documents  with  example 
results)  has  been  mailed  to  28  researchers  worldwide. 
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•  Results  of  the  test  bed  have  been  presented  at  the  WISE  2002  meeting  (May  2002,  Bergen,  Norway) 
and  at  the  ICCE  (Cardiff,  Wales) 

•  The  unsteady-state  test  bed  is  in  its  final  stages.  Most  of  the  observational  data  has  been  made 
available  by  Bob  Jensen  of  USACE,  Vicksburg.  At  this  moment  the  data  are  being  integrated  in  the 
test  bed.  This  task  should  be  finished  in  two  months  time. 

RESULTS 

For  example  results  we  refer  to  the  annual  report  of  2001. 

IMPACT/APPLICATIONS 

The  development  of  a  generic  community  suite  of  benchmark  tests  (agreed  upon  by  leading  model 
developers)  for  spectral  wave  models  will  promote  the  search  for  the  best  spectral  wave  model  and  the 
continued  improvement  of  wave  models.  It  provides  a  necessary  component  to  enable  researchers  and 
operational  institutes  to  evaluate  the  quality  of  new  (versions  of)  wave  models.  This  development 
agrees  well  with  the  current  policy  of  ONR  to  promote  the  development  of  a  community  wave  model  as 
a  common  frame  of  reference  for  all  related  ONR  projects.  Results  of  related  ONR-funded  research 
projects,  as  and  when  implemented  in  wave  models,  are  readily  assessed  with  the  proposed  benchmark 
suite. 

TRANSITIONS 

The  generic  community  benchmark  suites  with  all  their  data  and  documentation  are  freely  and 
unconditionally  available  (under  the  condition  of  proper  referencing  to  ONR  funding  and  to  the 
sources  of  information)  to  active  members  of  the  WISE  group  on  CD-ROM  (with  a  maximum  of  50 
copies  to  be  provided  for  distribution  by  ONR).  The  US  Navy  and  the  US  Army  are  active  WISE 
members.  Industrial  counterparts  are  welcome  to  use  the  benchmark  suite  free  of  charge  to  the  extent 
that  they  are  active  WISE  members.  If  an  institute  is  not  an  active  WISE  member,  the  suite  is  available 
on  CD-ROM  at  nominal  costs  from  WL|Delft  Hydraulics  for  a  period  of  two  years  after  the 
tennination  of  the  project. 

RELATED  PROJECTS 

Listed  below  are  various  projects  that  are  related  to  the  present  test  bed  project: 

1.  U.S.  Army  Engineer  Research  and  Development  Center,  Coastal  and  Hydraulics  Laboratory: 

Virtual  Test  Bed. 

2.  Oceanweather  Inc.:  Virtual  Test  Bed  for  Evaluating  Wave  Prediction  Technology. 

3.  Delft  University  of  Technology:  The  continued  development  of  the  third-generation  shallow  water 
wave  model  SWAN. 
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